1 research outputs found
P4Testgen: An Extensible Test Oracle For P4
We present P4Testgen, a test oracle for the P4-16 language that supports
automatic generation of packet tests for any P4-programmable device. Given a P4
program and sufficient time, P4Testgen generates tests that cover every
reachable statement in the input program. Each generated test consists of an
input packet, control-plane configuration, and output packet(s), and can be
executed in software or on hardware. Unlike prior work, P4Testgen is open
source and extensible, making it a general resource for the community.
P4Testgen not only covers the full P4-16 language specification, it also
supports modeling the semantics of an entire packet-processing pipeline,
including target-specific behaviors-i.e., whole-program semantics. Handling
aspects of packet processing that lie outside of the official specification is
critical for supporting real-world targets (e.g., switches, NICs, end host
stacks). In addition, P4Testgen uses taint tracking and concolic execution to
model complex externs (e.g., checksums and hash functions) that have been
omitted by other tools, and ensures the generated tests are correct and
deterministic. We have instantiated P4Testgen to build test oracles for the
V1model, eBPF, and the Tofino (TNA and T2NA) architectures; each of these
extensions only required effort commensurate with the complexity of the target.
We validated the tests generated by P4Testgen by running them across the entire
P4C program test suite as well as the Tofino programs supplied with Intel's P4
Studio. In just a few months using the tool, we discovered and confirmed 25
bugs in the mature, production toolchains for BMv2 and Tofino, and are
conducting ongoing investigations into further faults uncovered by P4Testgen